Typical computer systems, especially computer systems using graphical user interfaces (GUIs), are optimized for accepting user input from one or more discrete input devices, such as a keyboard for entering text, and a pointing device, such as a mouse with one or more buttons, for operating the user interface. An example of such a GUI is the user interface for the Windows® computer operating system (available from Microsoft Corporation of Redmond, Wash.). The ubiquitous keyboard and mouse interface provides for fast creation and modification of documents, spreadsheets, database fields, drawings, photos, and the like.
Recently, however, pen-based computing systems, such as tablet PCs and the like, have been increasing in popularity. In pen-based computing systems, user input advantageously may be introduced using an electronic “pen” or stylus (e.g., akin to writing with a pen or pencil on a piece of paper). Indeed, in at least some pen-based computing systems, all user input is capable of being entered and manipulated using an electronic pen input device, and the user interface is fully controllable using only the electronic pen.
As pen-based computing systems are becoming more popular, users are increasingly entering more data in the form of electronic ink. In many instances, however, users wish to convert the original electronic ink data to machine-generated text, text suitable for use and manipulation by conventional word processing programs and other application programs. In conventionally available systems, when ink is converted to text, it is converted as a block into text, and this new block of text typically replaces the original ink or exists separately as a text block. In such systems, virtually none of the original spatial information relating to the original ink arrangement and orientation is maintained (except for, possibly, line breaks), and much of the original user intent (e.g., larger writing size v. smaller writing size) is ignored. In some instances, correction of the converted text can be difficult, either because the original ink is difficult to call up and review after it has been converted and/or because the ink and text are located at two different positions.
Accordingly, there is a need in the art for improved electronic ink processing systems and methods that maintain the original spatial arrangement and orientation of ink and allow for easy review of the original ink during handwriting recognition and text correction. Such systems and methods would be particularly useful in pen-based computing systems and other devices that accept input in the form of electronic ink and/or from an electronic pen or stylus and/or systems or devices that display or otherwise process electronic ink data.